On the convergence of linear stochastic approximation procedures

Many stochastic approximation procedures result in a stochastic algorithm of the form 1 hk+l = h k + (bk A k h k ) , k for all IC = 1 , 2 , 3 . . . . . Here, { b k , IC = 1,2 ,3 , . . .} is a Rd-valued process, { Ak , IC = 1 , 2 . 3 . . . . } is a symmetric, positive semidefinite Redxd-valued process, and { h k k = 1,2 ,3 , . .} is a sequence of stochastic estimates which hopefully converges to l i m s u p P { z : 112 Qn(r)l1 2 S} 2 ~ / 3 ~ / q = ~ / 1 2 n-cc and therefore l i m ~ i i p , ~ ~ DT(Qn) > 0 for every T > 0 0 1 N m N -1 h b [$?E l V E4h] { lim E b k } (2) k = l k = l (assuming everything here is well defined). In this correspondence, we give an elementary proof which relates the almost sure convergence of { I i k , k = 1 , 2 , 3 , . . .} to strong laws of large numbers for { b k , k: = 1 , 2 , 3 ,...} and { A k , k = 1 ; 2 , 3 , . . . } . ACKNOWLEDGMENT The author wishes to thank R. Alexander for bringing the paper [ l ] to his attention.